Dirk Schnelle - Walka Technische Universität Darmstadt

My research interests lie in the area of Collaborative User Interfaces for Smart Spaces. Generally these interfaces are multimodal with a special focus on voice as one the major modalities. For me these interfaces should have a good user experience on the one hand and a good developer experience on the other hand. Dialog systems that are easy to implement most likely rely on a finite state machine or a frame based dialog system but usually result in rigid dialogs. Dialogs that appear to be more natural are harder to implement. For instance the Information State Update Model introduced by Larsson and Traum (Larsson and Traum, 2000) has the potential for very flexible and natural dialogs, but for the developer it is almost impossible to consider all variations of the defined update rules. Here, an approach to dialog modeling is needed which helps to minimize the developer’s efforts while keeping the potential to create more natural dialogs. Therefore, we settle upon existing standards like the W3C standard for Multimodal Architecture and Interfaces (Bondell et al., 2012) (MMI) and analyzed its expressiveness in comparison to existing architecural considerations of the past decades (Schnelle-Walka et al., 2013b). The architecture proposed by the W3C decomposes a multimodal application into a nested structure of interaction managers (IM) for dialog control and modality components (MC) for inand output. An application is conceived as a set of control documents expressed e.g. in SCXML (Barnett et al., 2012) for the interaction managers and a set of presentation documents with modalityspecific markup for the modality components. A topmost root controller document describes the global dialog and instantiates modality components as required. Each modality component can, in turn, again be an interaction manager, handling more fine granular concerns of dialog control, such as error correction or even sensor fusion/fission. In order to support the ISU approach, the dialog modeling IM ought to support grounding and reasoning for the information state (Fodor and Huerta, 2006). SCXML only features XPath and ECMAScript as embedded scripting languages, both lacking these capabilities. To solve this issue, we introduced Prolog as a scripting language for SCXML documents (Radomski et al., 2013), similar to the approach by Kronlid and Lager (Kronlid, 2007). Transitions and conditions are guarded by Prolog queries, enabling application developers to employ grounding and reasoning. Other language features are available to establish a-priori knowledge or introduce new facts as the dialog progresses. Eventless transitions, only guarded by conditions, are used to apply the update rules from ISU.